Introduces the split of the persistent state on to 2^persistent_state_split_depth files by shards. It supports write/import from splitted files, exchange via RPC, backward compatibility with a single-file persistent state.

It writes persistent state metadata file indicating the split config even when no split required.

<block_id>.meta.json
---
{
  "split_depth": 2,
  "parts": ["0xa000000000000000", "0x2000000000000000"]
}

Final persistent state bundle will be:

<block_id>_part_0xa000000000000000.boc
<block_id>_part_0x2000000000000000.boc
<block_id>.boc
<block_id>.meta.json

If state does not contain accounts in some shard, no part file is generated for that shard.

Crash recovery:

• write sequence: parts, metadata, main file
• if main file does not exist, bundle is not finished
• already written persistent state parts will be reused on the next attempt

Pull Request Checklist

NODE CONFIGURATION MODEL CHANGES

Added optional param. Default = 0

{
  "core_storage": {
    "persistent_state_split_depth": 2
  }
}

BLOCKCHAIN CONFIGURATION MODEL CHANGES

[None]

COMPATIBILITY

Fully compatible. Existing single-file persistent state treated as created with persistent_state_split_depth = 0.

SPECIAL DEPLOYMENT ACTIONS

[Not Required]

PERFORMANCE IMPACT

State 10kk.

• persistent_state_split_depth = 2

write start: Jun 19 11:31
write stop: Jun 19 11:49
elapsed: 18 min
size: ~1.4G (each part by 350M)

• persistent_state_split_depth = 0

write start: Jun 19 11:30
write stop: Jun 19 12:42
elapsed: 72 min
size: ~1.4G

TESTS

Unit Tests

Covered by:

• split_persistent_bundle_paths_are_rejected_for_zerostate_import
• persistent_state_meta_roundtrip
• shard_state_part_file_name_parser_recognizes_only_parts
• test_check_can_reuse_shard_state_part_files
• cache_rejects_split_sidecars_without_metadata
• persistent_state_writer_rejects_missing_absent_cell
• split_persistent_shard_state_import_from_dump
• overlay_server_split_persistent_state

Network Tests

[No coverage]

Manual Tests

Manual tests used
devnet5

• deploy 10kk
• init hack to write persistent on every key block on validator 2 and 3
• make a small change in blockchain config to force key block
• check persistent state creation